How many Laplace transforms of probability 
measures are there? 



FucHANG Gao* Wenbo V. Li^ Jon A. Wellner* 

January 1, 2010 



Abstract 

A bracketing metric entropy bound for the class of Laplace transforms of prob- 
ability measures on [0, cx)) is obtained through its connection with the small de- 
viation probability of a smooth Gaussian process. Our results for the particular 
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1 Introduction 



Let /X be a finite measure on [0, oo). The Laplace transform of z/ is a function on (0, oo) 
defined by 



m 



(1) 



It is easy to check that such a function has the property that {—l)"'f^^\t) > for all 
non-negative integer n and all t > 0. A function on (0, oo) with this property is called 
a completely mon otone function on (0, oo). A characterization due to Bernstein (c.f. 
WilhamsonI f ll956h ) says that / is completely monotone on (0, oo) if and only if there is 
a non-negative measure fi (not necessary finite) on [0, oo) such that ([T]) holds. Therefore, 
due to monotonicity, the class of Laplace transforms of finite measures on [0, oo) is the 
same as the class of bounded completely monotone functions on (0, oo). These functions 
can be extended to continuous functions on [0, oo), and we will call them completely 
monotone on [0, oo). 

Completely monotonic functions have remarkable applications in various fields, such 
as probability and sta tistics, physics and potential theory. The main properties of these 
functions are given in IWidderl ( 119411 ) . Chapter IV. For example, the class of completely 
monoto nic functions is closed under sums, products and pointwise convergence. We 
refer to Alzer and Bergj ( )2002l ) for a detailed list of references on completely monotonic 
functions. Closely related to the class of completely monotonic functions are the so- 
called fc-monotone functions, where the non-negativity of (—!)"/(") is required for all 
integers n < k. In fact, completely monotonic functions can be viewed as the limiting 
case of the fc -monotone funct ions as fc — »• oo. I n this sense, the present work is a partial 
extension of lCaol fl2008h and ICao and Wellnerl fl2009[ ). 

Let A^oo be the class of completely monotone functions on [0, oo) that are bounded 
by 1. Then 



/:[0,oo)^[0,oo) 



m 



-tx 



It is well known (see e.g. iFellerl f ll97ll ). Theorem 1, page 439) that the sub-class of Moo 
with /(O) = 1 corresponds exactly to the Laplace transforms of the class of probability 
measures fi on [0, oo). For a random variable with distribution function F{t) = P[X < 
t), the survival function S{t) = 1 — F{t) = P{X > t). Thus the class 



So 



S : [0,oo) ^ [0,oo) 



S{t) 



-tx 



is exactly the class of survival functions of all scale mixtures of the standard exponential 
distribution (with survival function e~*), with corresponding densities 



'o 



t > 0. 
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It is easily seen that the class Voo of such densities with p(0) < oo is also a class 
of completely monotone functions corresponding to probability measures /i on [0, oo) 
with finite fi rst moment. Thes e class e s hav e many applications in statistics; see e.g. 
Jewell! ( 1982 ) for a brief survey. Jewell! ( 1982 ) considered nonparametric estimation of a 



completely monotone density and showed that the nonparametric maximum likelihood 
estimator (or MLE) for this class is almost surely consistent. The bracketing entropy 
bounds derived below can be considered as a first step toward global rates of convergence 
of the MLE. 

In probability and statistical applications, one way to understand the complexity 
of a function class is by way of the metric entropy for the class under certain common 
distances. Recall that the metric entropy of a function class J-' under distance p is defined 
to be logA^(e, J-", p) where N{e,J-',p) is the minimum number of open balls of radius e 
needed to cover J-". In statistical applications, sometimes we also need bracketing metric 
entropy which is defined as log N[]{e , , p) where 



N[]{e,J^,p) :=min<jn : 3/^,/i,...,/^,/„ s.t. p(/„/J <£,^C [jU^J 

and 



k=l 



[l^,f,] = {geJ':l^<g<f,}. 



Clearly N{e,J^,p) < N[]{e,J^,p) and they are close related in our setting below. 

In this paper, we study the metric entropy of A^oo under the L^lu) norm given by 



I.H= / \f{x)Mdx), l<p<oo. 





where z/ is a probability measure on [0, oo). Our main result is the following 

Theorem 1.1. (i) Let u he a probability measure on [0, oo). There exists a constant C 
depending only on p > 1 such that for any < £ < 1/4, 

\ogN[]{e,Moc, II • \\lp(u)) < C^log(r/7) • I log£|2, 

for any < 7 < F < oo such that z/([7,r]) > 1 — 4^%^. In particular, if there exists a 
constant K > 1, such that u{[e^,e-^]) > 1 - 4~%p, then 

logA^[](e,A^oo, II ■ \\lp{u)) < CK\logef. 
(ii) If u is Lebesgue measure on [0, 1], then 

logiV[](£:, A^oo, II ■ Wl^u)) ^ logA^(£:, A^oo, || ■ ||l2h) >^ \ log£:|^ 

where B means there exist universal constants Ci,C2 > such that CiA < B < 
C2B. 

As an equivalent result for part (ii) of the above theorem, we have the following 
important small deviation probability estimates for an associated smooth Gaussian pro- 
cess. In particular, it may be of interest to find a probabilistic proof for the lower bound 
directly. 
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Theorem 1.2. Let Y{t), t > 0, be a Gaussian process with covariance EY{t)Y{s) — 
(1 - e-^-')/{t + s), then for < £ < 1 

logP ^sup < £^ x-|log£|^ 

The rest of the paper is organized as follows. In Section 2, we provide the upper 
bound estimate in the main result by explicit construction. In Section 3, we summarize 
various connections between entropy numbers of a set (and its convex hull) and small 
ball probability for the associated Gaussian process. Some of our observations in a 
general setting are stated explicitly for the first time. Finally we identify the particular 
Gaussian process suitable for our entropy estimates. Then in Section 4, we obtain the 
required upper bound small ball probability estimate (which implies the lower bound 
entropy estimates as discussed in section 3) by a simple determinant estimates. This 
method of small ball estimates is made explicit here for the first time and can be used 
in many more problems. The technical determinant estimates are also of independent 
interests. 



2 Upper Bound Estimate 

In this section, we provide an upper bound for A^[](£, Aloo, || • IUp(;/))5 where is a 
probability measure on [0, oo) and 1 < p < oo. This is accomplished by an explicit 
construction of ^-brackets under Lp^v) distance. 

For each < £ < 1/4, we choose 7 > and F = 2"*7 where m is a positive integer 
such that i^([7, F]) > 1 — 4~"^£^. We use the notion I(a <t<h)to denote the indicator 
function of the interval [a, b). Now for each / e M.00, we first write in block form 

m 

fit) = 1(0 < i < 7)/(i) + l{t > r)f{t) + 5^I(2-S < t < 2^1) fit). 

1=1 

Then for each block 2*^^7 < t < 2*7, we separate the integration limits at the level 
2^~*|log£|/7 and use the first N terms of Taylor's series expansion of e~" with error 
terms associated with ^ = Cu,N: < ^ < 1, to rewrite 

m 

fit) = 1(0 < i < 7)/(i) + Kt > r)fit) + J2(Pi(t) + + W) 

i=l 

where 

Piit) =: 1(2^-^ < t < 2^7) J2 T / x^'nidx) 

q,it) =: I(2^-S < t < 2^7) I iN + l)\ ^^^^^ 

POO 

nit) =: I(2'-^7 < t < 2^7) / e-*^//(dx). 

i22-i|log£|/7 
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We choose the integer so that 



4e^| logel - 1 < < 4e^| logej. 



(2) 



Then, by using the inequahty k\ > {k/eY and the fact that < ^ < 1, we have within 
the block 2*-^7 < t < 2^7, 



(N + iy. 



41oge|^+^ /4e|log£: 



< 



(iV+1)! 



< 



+ 1 



N+l 



where we used tx < 2*7 ■ 2^ *| log£|/7 = 4| log el in the second inequality above. This 
implies, due to disjoint supports of qi(t), 



i=l 



< e 



4e^ 



Next, we notice that for t > 2* 7 and x > 2 *7 | loge|, e < e . Thus 



2=1 



< V 1(2'^-^ < t < 2*7) / e^fi{dx) < 

i=l J22-'7-i|log£| 



Finally, because |/| < 1 and z/([0,7)) + z/([r,oo)) < 4 we have 
Together with (|3]) and (jl]), we see that the set 



(3) 



(4) 



4 = 1 



i=l 



has diameter in LP(z/)-distance at most + e^'^^ + e/4 < e/2. 

Therefore, if we denote Vi = {pi(t) : / G A^oo}, then the expansion of / above 
implies that Ai^o C. J2iLi T^i + ^5 consequently, we have 



N^l{e,Moo, II ■ IIlph) < iV[] e/2,J2'P^, II • lU^ 



i=l 



For any 1 < i < m and any pi EVi, we can write 



p,(t) = I(2*"S < t < 2*7) E(-l)"«™(2"'7"'t)' 



(5) 



n=0 
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where < a„j < |41oge|"/n!. Now we can construct 



N 

n=0 
N 

= I(2'-i7 < t < 2^7) ^(-l)"c„,(2-7-it)-, 

n=0 



where 



_p^_a^^ n is even [ n is even 

_^^2:^^j ^ is odd I 2^^r^^^^l n is odd 

Clearly, p.{t) < pi{t) < p^it), and 

N 

\p,-p^ < I(2-S<t<2S)5^|c™-6™|(2-VH)" 



n=0 

< I(2-S<t<2^7)5^^(2-V'tr 

n=0 



< -1(2-^7 < i < 2''7)- 



Hence 



J]p,<5^P.<5^P.<5^P, + e/2. 

1=1 1=1 i=l i=l 

That is, the sets 

{m ^ f m 

y^^p. '■ Pi G Vi, 1 < z < m > and T-^ =■ \ ^^Pi • Pi ^ 1 < i < m 

form e/2 brackets of Yl^i'^i Iv°°-norm, and thus in L^(z/)-norm for all 1 < p < oo. 

Now we count the number of different realizations of V and P. Note that, due to 
the uniform bound on a„j in there are no more than 

2"+i 14 log el" 

^ + 1 

e nl 

realizations for bni- So, the number of realizations of pj is bounded by 

n=0 ^ ' ' 

Because n\ > (n/e)", for all 1 < ri < iV, we have 

2"+i |41oge|" ^ ^ ^ 3 /8e|log£| 



e n\ e \ n 



Thus, the number of reahzations of is bounded by 



\ N+l / N \ 

1 ■ exp ( y~^(nlog |8elog£| — nXogn) ) 

N+i / i\Tii\T , 1 \ rN 



?,\"^' /N{N + 1 



< \~j ■ ^ 2 log |8elog£:| — y xlogxdx 

< I -I -exp I log|8elog£:| - — logA/ + — 

< exp(C|logep) 

for some absolute constant C, where in the last inequality we used the bounds on 
given in ([2]). 

Hence the total number of realizations of V is bounded by exp {Cm\ logep). Similar 
estimate holds for the total number of realizations of P, and we finally obtain 

logA^[](e, A^oo, II ■ IIlph) < C'm\\oge\'^ 
for some different constant C. This finishes the proof since m = log2(r/7). 



3 Entropy of Convex Hulls 

A lower bound estimate of metric entropy is typically difficult, because it often involves 
a construction of a well-separated set of maximal cardinality. Thus we introduce some 
soft analytic arguments to avoid this difficulty and change the problem into a familiar 
one in this section. The hard estimates are given in the next section. 

First note that A^oo is just the convex hull of the functions ks{-), < s < oo, where 
ks{t) = e~**. We r ecall a general method about the entropy of convex hulls that was 



introduced in iGad (120041 ). Let T be a set in or in a Hilbert space. The convex hull 



oo 



of T can be expressed as 

{oo 
cintn : tn e T, a„ > 0, n G N, ^ a„ = 1 
n=l n=l 

while the absolute convex hull of T is defined by 

{oo oo 
ttntn : tn G T, n G N, |a„| < 1 > . 
n=l n=l J 

Clearly, by using probability measures and signed measures, we can express 



conv(T) ~ t^{dt) : yU is a probability measure on T 
abconv(T) = { \ tii{dt) : /i is a signed measure on T, ||/i||Ty < 1 



T 
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For any norm || • || the following is clear: 

conv(T) C abconv(T) C conv(T) — conv(T). 

Therefore, 

A^(e,conv(T), || ■ ||) < A^(e, abconv(T), || • ||) < [iV(e/2, conv(r), || ■ \\)]\ 

In particular, at the logarithmic level, the two entropy numbers are comparable, modulo 
constant factors on e. The benefit of using absolute convex hull is that it is symmetric 
and can be viewed as the unit ball of a Banach space, which allows us to use the following 
duality lemma of metric entropy: 



log A^(£:, abconv(T), || ■ | 
where B is dual ball of the norm || ■ ||, and 

||x||t := sup I {t, x) I 



: \ogN{c2e,B, \\ ■ \\t) 

\t is the norm introduced by T, that is. 



sup 

tGabconv(T) 



{t,x) 



Strictly speaking, the duality lemma remains as a conjecture in the gener al case. How- 



ever, when the norm || ■ | | is the Hilbe rt space norm, this ha s been proved. See lTomczak-Jaegermann 
(1l987h . lBourgain et al.l f ll989h . and lArtstein et all fl2004j| . 

A striking relation discovered by lKuelbs and Lil (119931 ) says that the entropy number 
log A^(£:, B, II ■ \\t) is determined by the Gaussian measure of the set 

=: {x e H : \\x\\t < e} 



under some very w eak regularity assumptions. For details, see iKuelbs and Lil ( 1l993l ). 



Li and Lindd (119991 ) . and also Corollary 2.2 of lAurzada et al.l ( l2008l ). Using this rela- 



tion, we can now summarize the connection between metric entropy of convex hulls and 
Gaussian measure of into the following 

Proposition 3.1. Let T be a precompact set in a Hilbert space. For a > and /3 G M, 



logP(D,) < -Ci£-"|log£ 



/3 



if and only if 



log A^(£:, conv(T), 



2/3 



> 2+" I log el 2+« : 



and for /3 > and 7 G 



if and only if 



logP(De) < -Ci|l0g£|^(l0g|l0g£|)' 



logA^(e,conv(r),|| ■ II2) > C2I loge|'^(log | loge|)^ 
Furthermore, the results also hold if the directions of the inequalities are switched. 
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The result of this proposition can be imphcitly seen in iGad (120041 ). where an expla- 
nation of the relation between N{e,B, || ■ and the Gaussian measure of is also 
given. 

Perhaps, the most useful case of Proposition 13.11 is when T is a set of functions: 
K{t, ■), t & T, where for each fixed t eT, K{t, ■) is a function in L^(fi), and where fl is 
a bounded set in Mf^, d > 1. For this special case, we have 

Corollary 3.2. Let X{t) = J^K{t,x)dB{x), t e T, where K{t,-) are square-integrable 
functions on a bounded set Q in R"', d > 1, and B{x) is the (i- dimensional Brownian 
sheet on Q. If J-' is the convex hull of the functions K(-, u), u E Q, then 



logP ( sup \X{t)\ < e 



£-°|log£|^ 



for a > and /3 G M if and only if 

logN{e,T, 

and for /3 > and 7 G M, 



2a 2(3 

e 2+Q loee 2+" 



logP ( sup|X(t)| < e 



— I log£:|^(log I logel)' 



if and only if 



\ogN{e,J', II ■ II2) X I log£:|^(log| log£:|)^. 



The authors foun d thi s coro llary especially useful. For example, it was used in 



Blei et al.l ( 120071 ) and iGaol (120081 ) to change a problem of metric entropy to a problem 



of small deviation probabilit y of a prob lem about a Gaussian process ^ yhich is relatively 
easier. The proof is given in Gao ( 2008 ) for the case = [0, 1], and in lBlei et al.l ( 2007 ) 
for the case [0, l]'^. For the general case, it can be proved as easily. Indeed, the only 
thing we need to prove is that P(-De) can be expressed as the probability of the set 
sup^gT^ 1^(^)1 < ^- We outline a proof below. Let 0„ be an orthonormal basis of L^(f2), 



then 



X(t) 



/ K{t,s)dB{s) = y2^n / K{t,s)(j)n{ 
Jn Jn 



s)ds 



where ^„ are i.i.d standard normal random variables. Thus, 



f{s)g{s)ds 



<ej 




K(t, s)g{s)dsfi{dt) 



IT Jn 

00 



P < «n0n(s) : ^ < 00, 



. n=l 

00 



n=l 

00 



n=l 




K{t, s)(f)n{s)dsfi{dt) 



T Jn 



P S ^ an0n(s) : ^ < 00, sup 



n=l 



n=l 



teT 



00 „ 

J2^n K{t,s) 
n=l 



{s)ds 



< e 



P ( sup \X{t)\ < e 

t&T 



Now back to our problem of estimate \ogN{e,Aioo, \\ ■ II2) in the statement of (ii) 
of the theorem, where || ■ II2 is the norm under the Lebesgue measure on [0,1], we 
notice that Aioo is the convex hull of the functions C(-,s), s G [0, cxd), on [0,1] with 
C{t,s) = e"*"*. However, [0, 00) is not bounded. In order to use CoroUarv 13.21 we 
need to make a change of variables. Notice that by letting y = e"**, we can view A4oo 
as convex hull of K{-,y), y G (0,1], where K{t,y) = y*. Clearly, K(t,-) are square- 
integrable functions on the bounded set (0, 1]. Now, for this K, the corresponding X{t) 
is a Gaussian process on [0, 1] with covariance 

EX(t)X(s) = s,tG(0,l], (s,t)^(l,l), 

and EX(1)^ = 1. Thus, the problem becomes how to estimate 



P I sup \X{t)\ < £ , or equivalently P sup \Y{t)\ < e 

^iG(0,l] J \t>o 

where Y{t) = X(e~*), which has covariance structure 

EY{t)Y{s)= \ s,t>0. (6) 

We now turn to the lower estimates of this probability. 

4 Lower Bound Estimate 

Let Y{t), t > Ohe the centered Gaussian process defined in ([6]). Our goal in this section 
is to prove that 

logP(sup \Y{t)\ <e) < -C\ log£|^ 

t>o 

for some constant C > 0. 

Note that for any sequence of positive numbers {Si}^^^, 

P (sup \Y{t)\ <e] < P(max \Y{S,^\ < e) 



t>o 



l<i<n 



= (27r)-"/2(detS)-i/2 /" exp{-{y,J:-'y))dy,---dyn 

^maxi<i<„ \yi\<£ 

< (27r)-"/2(detS)-^/2(2£)n 

< £"(detS)-i/2. (7) 



where the covariance matrix 

'1 _ 



S = (KY{6,)Y{5,)) 



l<M<n \^ 5+ A. 

^ ' ' J / l<t,j<n 



To find a lower bound for det(S), we need the following lemma: 
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Lemma 4.1. If < bij < aij for all 1 < i, j < n then 

" b 

det{aij — bij) > det(ajj) — > max — • per( 



aij). 



where per(ajj) is the permanent of the matrix (aij). 
Proof. For notational simplicity, we denote Cij = aij — bij, then 
det {aij — bij) — det(ajj) 

= — l)'^Ci^CT(i)C2,<j(2)Cn,a(n) — — l)'^ai,(T(l)a2,<T(2) " " " On,<T(n) 



a k=l 
n 

> — ^^['3i,(t(1) ■ ■ ■ (lk-l,a{k-l)]{bk,a(k))[0'k+l,a{k+l) ' ' ' (ln,a(n)] 



a k=l 



> — max — y^[ai,cr(l) ■ " ■ Cik~l,CT(k-l)]icik,a(k))[<^k+l,a(k+l) ■ ■ ■ 0.n,a(n)] 

' KKn ai-i '—^ 



k=l ""^^ 

bki 



— max — ■ per(^ajj^ 



i<l<n akl 
fc=l 



In order to use Lemma WA\ to estimate det(S), we set 

aij = ^ ^ ^ , and bij = e~^'~^'aij 
' 6i + 6j 

for a specific sequence {6i}^^^ defined by 

5mp+g = 4P+™(m + g), 0<p<m,l<q<m 

for n = m^. 

Clearly, we have 

< bki/aki < e-^'"^'", l<k,l <n = m^. 
It remains to estimate det(aij) and per(aij), which is given in the following lemma. 

Lemma 4.2. For the matrix {aij) defined above, we have per(ajj) < 1, and det(ajj) 
(240e)-2"^'. 



Proof. It is easy to see 



(m^)! 

perfaj,) < n!(maxaj,)" < — ^ — r < 1 
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since < {2m4"^) ^ for 1 <i,j < 



n = 711?. 



To estimate det(ajj), we use the Cauchy's determinant identity, see iKrattenthaler 



mm . 



ni<i<,<„('^i - ^i? 1 f^i- Si 



2 



det(a,.) - det ^ ^ J - y^^^^ + - 3. n^., 6. ' „ V^. + 5. 

For l<'i<j<n = m^, write i = mp + g and j = mr + s with 1 < g, s < m. Denote 

y4 = {{i,j) '■ i = mp + q,j = mp + s,0 < p < m — 1,1 < q < s < m}, 

B = '. i = mp + q,j = m{p + l) + s,0<p<m — 2,1 < q,s < m}, 

C = '■ i = mp + q,j = mr + s,0<p<m — 3,p + 2<r<m — l,l<q,s< m}. 

Then A, B and C form a partition ofl<'i<j<n = m^. 
Next we estimate each part separately. First, for p = r, 

6j — 6i s — q s — q 

— > 



Thus 



Sj + Si 2m + s + q 4m 



n n ^ -uuil 

Vi,j;t^ ■' ^ P=0 l<q<s<m ^ ^ k=\ q=\ ^ 



> 



k \ I \m — l)\ 



n 



k=\ 

> (8e) 



Am J \(4m)'"-i 

2m3 



Second, for r — p = 1, 



Thus we have 



Sj - Si _ (4m + 4s) - (m + g) ^ 1 
Sj + Si~ (4m + 4s) + (m + q) ~ 5' 



r X \ 2 m— 2 

'^j ~ "i\ ^ TT TT c;-2 ^ e;-2m3 



n iBi) n s-^^- 

Third, for r — p > 2, 



, 5i + 5i , 



- (5A _ 4''(m + s) - 4P(m + g) _ ^ 2 ■ 4P(m + g) ^ 

\Si) ~ 4'"(m + s) + 4P(m + g) ~ 4''(m + s) + 4P(m + g) 
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Thus we have 

((- X \ 2 m— 3 m— 1 , , \ 2 m— 2 

-t^) a n n n (i-i^) anii-^-T' 

Vij;tv^ ^ *^ p=0 r=p+2 l<g,s<m ^ ^ fc=l 

fc=l z=i 



> exp (-2,„» g j > exp (-2™» g 1 j 



Therefore, we have 



exp j 


^-2m' 


exp 


-2m' 


(2/3)""' . 




n 







5i — bi \ -n- -n- -TT / (5i — 



2 



-2m 



n ^ - n-n-n 

On the other hand, it is not difficult to see that 

n m m— 1 

2^^ JJ 5, = 2"^' JJ JJ 4?'+'"(m + g) < 2"*' ■ 4'"'(™-i)/2+r«3 (2m)"' 

j=l (j=l p=0 

^3m^/2+m^/2+m^ log4 m ^ ^2m^ 

for m > 1. Therefore, 

2 



3 



det(a.,)= (2"n5,) ■ H (t^) > (240e) 

V i=\ j l<i<j<n V J 



-2m'^ 



Now combining the two lemmas above, and using the estimate in (|8]), we obtain 
det(S) > (240e)-2™' _ ■ e""'"^'" > e"^^'"' 
provided that m is large enough. Plugging into ([7]), we have 

p(sup|F(t)| <e] < e^"^'^™'. 
\t>o J 

Minimizing the right-hand side by choosing m ^ \ log£:|/12, we obtain 

P (sup|F(t)| < £ ) < exp (-(432)"i|loge|^) . 
\t>o J 

Statement (ii) of Theorem 11.11 follows by applying Corollary 13.21 At the same time, 
also finished the proof of Theorem 1.2. 
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